Bayesian Estimation of Dynamic Discrete Choice Models

We propose a new estimator for dynamic programming discrete choice models. Our estimation method combines the Dynamic Programming algorithm with a Bayesian Markov Chain Monte Carlo algorithm into one single Markov Chain algorithm that solves the dynamic programming problem and estimates the parameters at the same time. Our key innovation is that during each solution-estimation iteration both the parameters and the expected value function are updated only once. This is in contrast to the conventional estimation methods where at each estimation iteration the dynamic programming problem needs to be fully solved. A single dynamic programming solution requires repeated updates of the expected value functions. As a result, in our algorithm the computational burden of estimating a dynamic model is of similar order of magnitude as that of a static model. Another feature of our algorithm is that even though per estimation iteration, we keep the number of grid points on the state variable small, we can make the number of effective grid points as large as we want by simply increasing the number of estimation iterations. This is how our algorithm overcomes the “Curse of Dimensionality.” We prove that under mild conditions, similar to those imposed in standard Bayesian literature, the parameters in our algorithm converge in probability to the true posterior distribution regardless of the starting values. We show how our method can be applied to models with standard random effects where observed and unobserved heterogeneities are continuous. This is in contrast to most dynamic structural estimation models where only a small number of discrete types are allowed as heterogeneities.